Ham Radio 2000

home *** CD-ROM | disk | FTP | other *** search

/ Ham Radio 2000 / Ham Radio 2000.iso / ham2000 / tcp_ip / gp / 7plus.eng < prev next >

Wrap

Text File | 1992-01-02 | 36KB | 989 lines

.***************************************************************** *** *** *** 7PLUS - A new and efficient coder/decoder for transfer of *** *** any kind of data through store & forward nets. *** *** *** *** Version 1.50 Axel Bauda, DG1BBQ @DB0CL (09/25/91) *** *** Potsdamer Str.43 *** *** W-2850 Bremerhaven 21 *** *** Germany *** *** *** *** 7PLUS is Public Domain. It may be copied freely. The DOC- *** *** files and 7PLUS.C must be included! This document is valid *** *** for the PC-version as well as for the AMIGA-, ATARI_ST- *** *** and UNIX-versions *** *** *** ***************************************************************** The not-reading-the-manual-phobia (antidocphobia): ================================================== I always happens again. People asks me questions that were already dealt with in this manual. I know, it's hard for some people to read the manual thoroughly, but it saves me a lot of unnecessary work. So in case of trouble, read the manual very carefully. I have gone through great length to make this manual as detailed as possible. However, I may have forgotten a few points here an there. Nobody's perfect...... I cannot supply a MS_DOS/ATARI/AMIGA or UNIX course in this manual. Therefore you should be familiar with the basics of your computer. Also, you should be familiar with the handling of your terminal program! Preface: ======== All present coders like 7BIT, RADIX95, UUENCODE and SEVEN do not consider the fact, that files can get corrupted upon transfer in packet radio nets. Up to now, corrupted files had to be repeatedly sent to get them trough correctly, if ever. This is where 7PLUS gets in by automating the repair of corrupted files as far as possible and eliminating the need of retransmission of entire files. Automation and reliable recognition of corrupted lines are made possible by adding cyclic redundancy checking and line numbering to each line of code, therefore allowing exact protocolling of errors. Let's say OM A has encoded TEST.TXT to TEST.7PL and uploads it to a BBS. OM B downloads it and tries to decode it. ..Unfortunately, the file has been corrupted.... 7PLUS detects the faulty lines and creates an error report called TEST.ERR. OM B sends TEST.ERR to OM A, so he can create the correction file TEST.COR with the help of 7PLUS. OM A then uploads TEST.COR to the BBS. Providing OM B received TEST.COR uncorrupted (in that case TEST.COR would have to be resent), he can have the corrupted TEST.7PL regenerated and decode it successfully. In case TEST.7PL contains more errors than are covered in TEST.COR, only a new TEST.ERR needs to be generated and to repeat the above procedure. That's still much more effektive than resending the entire file. I admit, the whole concept may sound somewhat complicated at first glimpse. It isn't really. The described procedure is just the realization of the three basic steps: 'Dectect errors', 'Tell Originator about it and get correction info', 'Do correction'. Once you understood the concept and have dowloaded programs successfully, you will see that it isn't that complicated. The development of 7PLUS: ========================= 7PLUS has been developed on the basis of my previous encoder SEVEN. The correcting capabilities had the price of incompatibility to SEVEN, however. This isn't so grave, since 7PLUS has a lot of advantages compared to SEVEN and now the repeated upload of files, that were corrupted in store & forward, is a thing of the past. Since 7PLUS was incompatible to SEVEN anyway, I implemented a RADIX216 encoding wich allows for very compact files. In the beginning, there was heavy opposition against 7PLUS. Some people were convinced that an encoder that uses characters above 7Fh would create more trouble than do good. 7PLUS established itself as THE standard encoder in Germany very quickly. This proves the critics to be wrong. By now, 7PLUS is not only used in Germany, but in the greater part of Europe. Even in Japan, 7PLUS has already been 'sighted'. There even are BBSs that use 7PLUS for file serving purposes. Page - 2 How does the 7PLUS encoding work? (roughly) =========================================== 7PLUS implements a RADIX 216 encoding algorithm which only produces a file enlargment of 3.23%. This algorithm encodes 31 input bytes into 32 ASCII bytes. It could also be called 7-threequarter-bit- encoding, because each digit represents that many bits of the unencoded data. That's why it's called 7PLUS. Line by line checksums, line numbering, line separators, header and foot lines increase the enlargment ratio to 11 to 12%. Fileenlargments of other coders: RADIX95 : 24% UUCODE : 42% Each digit of the resulting code has a value range of 0 trough 215. Since this range can't be directly transmitted (contains control characters), the code is transposed to avoid any problematic characters. The resulting character set is as follows: 21h - 29h, 2Bh- 7Eh, 80h - 90h, 92h, 94h - FCh (including) These characters are left out intentionally: 00h bis 20h: Control codes and the space character. Somes terminal eliminate spaces at the end of lines! 2Ah : Asterisk, may cause problems with some BBSs. e.g.'*** end' 7Fh : DEL, another control character. 91h und 93h: Some phone terminals interpret these codes as XON and XOFF...for whatever reason??? Probably because of an error. The only factor that can create problems with 7PLUS-files is the fact, that characters above 7Fh are used. Some terminalprograms do not handle these characters. Experience in Germany and the surrounding countries has shown that all BBSs (TheBox, W0RLI, F6FBB, AA4RE...) are capable of handling these characters and the greater part of terminalprograms as well. Already, terminalprograms are being updated to cope with the character set used by 7PLUS. TNCs do not pose a problem, because they can all be configured to handle characters above 7Fh (8BITCONV ON, AWLEN 8....). It won't take long until all terminalprograms used in packet radio will be capable of handling 7PLUS-files. If 7PLUS comes into wide use like in Germany, authors of terminals will (have to) update their software to comply with the need of data reduction. Page - 3 How are 7PLUS-files made up? ============================ 7PLUS-files always start with the header line, followed by up to 512 lines of code and one foot line indicating the end of the file. All lines are terminated with a line separator. During transmission via packet radio, the separator is the CR character. On Disk, the separator must comply with the individual system (CR/LF on IBM-PC and ATARI ST, LF on AMIGA/UNIX). The header: ----------- go_7+. 005 of 010 TEST.TXT 0010453 6488 166 (7PLUS v1.5) XXXXY +-----+ +-+ +-+ +----------+ +-----+ +--+ +-+ +----------+ +---+ 1 2 3 4 5 6 7 8 9 1) Start indicator, beginning in the second column. 2) Part 5 3) of 10 parts. 4) Filename of the encoded File, filled up with spaces. Max 12 characters. On systems that allow longer filenames, still an MS_DOS-compatible filename is sent here! 5) File length of the unencoded file 6) Number of ASCII-Bytes in this part (HEX) 7) Blocksize (HEX) 8) Codername and version. 9) XXXX: Testcharacters (to detect stripping of eigth bit). Also used to indicate that the full filename follows in the next line (only v1.4 and above). This is only the case with the AMIGA/UNIX-version currently. This way, exchangeability of 7PLUS-files that were created on systems with different filename structures than MS_DOS is insured. Y: Minichecksum for the header (7PLUS v1.5 and above). The code lines: --------------- AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBPPP +--------------------------------------------------------------++-+ 64 ASCII bytes, representing 62 binary bytes * A: First group (32 ASCII bytes, representing 31 binary bytes). B: Second group. P: Checksum and line numbering. Every line is made up of 64 ASCII bytes plus 3 bytes, that contain the checksum and line number. The three bytes at the end of each lines are made up of a 14 bit CRC and a 9 bit linenumber. Page - 4 Each 7PLUS-file contains a maximum of 512 code lines. If more are needed, the file is splitted. Thus, each 7PLUS-file can contain up to 512x62 = 31744 bytes. With a maximum of 255 parts, 8,094,720 bytes can be handled. This should be enough for every need. The foot line: -------------- stop_7+. (TEST.P05) +-------+ +--------+ 1 2 1) End indicator, beginning in the second column. 2) Filename of this part. The error report: ================= In the case that 7PLUS-files were damaged in transit, 7PLUS recognizes the errors and creates an error report that contains all necessary information for creating an according correction file. An error report looks like this: go_text. TEST.ERR 7PLUS error report: TEST.TXT 113 /Test.TXT/ 34028 01 005 006 007 03D FFF 02 10C 10D 10E FFF 00 CRC A17D stop_text. The lines 'go_text.' und 'stop_text.' are used to recognize and extract the error report from a capture file. The line '7PLUS error report...' contains information like: Name of the original file, blocksize in wich the file was split while encoding, the full filename (only AMIGA/UNIX) and (from 7PLUS v1.5) the length of the original file. The two-digit hex values define the part in wich the following lines were corrupted. A line number of 'FFF' is used to mark the end of the error list for this part. '00' for the part marks the end of the error report. In the next line a checksum for the error report (7PLUS v1.5 and above) follows to secure it additionally. Page - 5 The correction file: ==================== After you have sent the error report to the originator of the corrupted files, he will use it to create a correction file, that contains all lines that were reported as damaged or missing. He then sends the correction file back or uploads it to the BBS if he sees that most error reports concern the same errors. In that case the '-J' option may come in handy. He could join different error reports and create a global correction file. Example of a correction file: (Contents of the code lines shown in brief) go_text. TEST.COR 7PLUS correction: TEST.TXT 34028 P01: L005: sdWKLRjwEK... L006: "rt4rWER7Z... L007: eeERG35ERG... L03D: $%&H$%6TZJ... P02: L10C: AdD$%SFFDr... L10D: sFG$sFw4rf... L10E: 56fZTddfDg... P00: CRC A9B6 stop_text. 'go_text.', 'stop_text.' and checksum: see 'The error report'. The line '7PLUS correction...' contains the following information: Name of the original File and (7PLUS v1.5 and above) the length of the original file. A line with an entry of the from ' PXX:' defines for which of the encoded files the following lines are meant. The coded lines are preceded by ' LXXX:' wich states the line number in the file. 'XX' and 'XXX' stand for two- and three-digit hex values. The structure of the error reports and correction files is intentionally held very simple. Page - 6 Compatibility of 7PLUS versions: ================================ All files of the older versions can be decoded with the newer versions and vice versa. This also counts for error reports and correction files. Beginning with 7PLUS v1.5, a checksum is added to the error reports an correction files. It is only checked when present. When 7PLUS v1.5 and newer gets files from an older version it outputs a warning message, that no checksum was found: 'no CRC. File may be corrupted or from version earlier than 7PLUS v1.5' However, 7PLUS will continue processing the files. The warning only means that the file may not be 100% correct. You should check those files visually for obvious errors. Since the structure of the error reports and correction files is very simple, it is easy to erase the faulty lines. Mind, that if a checksum is present (CRC XXXX), it must be erased, otherwise the file would not be accepted by 7PLUS anymore! Filename conventions: ===================== Filenames handled by 7PLUS, must comply with the rules concerning filename generation under MS_DOS, no matter what system 7PLUS is implemented on. They are always made up of a name (8 charachters max.) and an extension (3 charachters max.) seperated by a period (.). Filenames in 7PLUS-files are always in uppercase. On disk, lower case is used for the filenames, if the filesystem allows this. This is very important on systems that do not ignore the case in filenames (UNIX). E.g.: TERMINAL.EXE To enable transfer of full filenames on systems with name structures different than MS_DOS, the full filenames are transfered separately (see 'The header:'). At the same time, a MS_DOS compatible filename is created to insure exchangeability. E.g., the full filename 'Hyper_Term_Plus' would be changed into 'HYPER_TE'. When encoding a file without splitting it, the encoded file is assigned '.7PL' as the extention. (TERMINAL.7PL) Split files are assigned '.PXX' as the extension, where XX is a hexadecimal number in the range 01 through FF. Error reports usually have '.ERR' and correction files always have '.COR' for an extension. This is very important, because 7PLUS gets it's information on what to do from the extension! It Page - 7 is allowed to change names of error reports and correction files. However, the extensions MUST be '.ERR' or '.COR'. If you recieved a lot of error reports, there is another way of naming the error reports. The first report will have '.err' for an extension and the following ones will have '.eXX', where XX ist a two-digit hex value. Example: You received three error reports that all relate to the same upload (TEST.7PL). When extracting through the '-X' option, the first report will be called TEST.ERR, the second TEST.E01 and the third TEST.E02. The three reports can then easily be joined into a combined report using the '-J' option. 7PLUS-files should always be stored with filenames corresponding to these rules. This is even more important in regard to split files and correcting. The name under wich a 7PLUS-file should be stored can be found at the end of each part (see 'The foot line'). How do i use 7PLUS? =================== Encoding: --------- 7plus test.txt Encode test.txt. If the resulting file contains more than 512 lines of code, the file is automatically split into 512 line portions. 7plus test.txt -s Encode and split, 140 lines per file (abt 10k/file). 7plus test.txt -s 30 30 lines per file. 7plus test.txt -sp 3 Encode and split into three files of roughly equal size. Decoding: --------- 7plus test.7pl Decode. Create error report, if test.7pl is corrupted. 7plus test.p01 Combine and decode. " 7plus test When decoding, the extension may be ommited. If 7PLUS finds test.cor, it will do the correction first, and decode after- Page - 8 wards. If a file named 'test' already exists, it will be encoded! Correcting: ----------- 7plus test.err c:\pr\ Compile correction file test.cor. Look for test.txt in c:\pr\. test.txt is the original binary file that was encoded in the first place. It is not necessary to keep the encoded files after uploading them. But you should keep the original file (test.txt in this case) to be able to reply to error reports! A special situation arises, when a 7PLUS-file was created on a system with non-MS_DOS filenames. Example: An OM encodes 'Hyper_Term_Plus' on an AMIGA. 7PLUS will create the MS_DOS-kompatible name 'HYPER_TE' (The full AMIGA-filename ist also passed along in the file, but the PC and ATARI_ST version ignore it). If someone decodes the file using an older version of 7PLUS (before 1.4) and errors are present in the file, the error report 'HYPER_TE.ERR' will be created. If he sends it to the AMIGA-user, 7PLUS will try to find 'HYPER_TE' while creating the correction file and will probably not find it. In this case, the full path and filename can be specified: Example: 7plus hyper_te.err df0:utils/Hyper_Term_Plus If 7PLUS v1.4 (or newer) was used, this problem does not arise, because the full original filename is passed along with the MS_DOS compatible filename in the error report - even if it isn't evaluated otherwise on the PC and ATARI_ST version. 7plus test.cor Repair test.7pl/test.pXX, using test.cor. It may happen, that not all errors are covered by the correction file. In that case decode the files to create a new error report. Send it to the originator for additoinal correction. 7plus test.err test2.err -j The contents of test2.err are added to those of test.err. this ist useful, when many error reports were recieved. You can create a combined error report that in turn can be used to compile a correction files that covers all reported errors. Mind that error reports that contain an unusually large number of error lines should be answered individually! 7plus test.err -j This is a special form of joining error reports. 7PLUS will look for test.e01 first and join it to test.err, then for test.e02 and so on.... It is important for this, that the error reports were extracted from a capturefile using the '-X' option! After this action test.err will represent the sum of all error reports. Page - 9 7PLUS-file extractor: --------------------- 7plus capturefile text -x 'cuts' all 7PLUS-files from a capturefile and saves them seperately. if a text is specified, only those files will be extracted, that contain text in their names. E.g.: 7plus spsave.c10 952.err -x Extracts all error reports that have '952.err' in their filenames. If a fictive text is specified (e.g. '###'), no files are extracted, but you get a list of the files contained in the capturefile. It is irrelevant, what kind of line seperator is contained in the capturefile: CR/LF, CR or LF. The extracted files however will have line seperators that comply with the used system. That is CR/LF on IBM PC and ATARI ST, LF on AMIGA/UNIX. This way, there is no need to use an editor to cut out the files from the logfile and correct generation of filenames is guaranteed. This is particularly important with split files. Also, error reports and correction files are extracted. This will only work, if the markings (go_text., stop_text.) are present! Additional options: ------------------- -N (only ATARI_ST version) Normally 7PLUS prompts for a key hit after termination. The '-N' option will switch off this prompt. This is useful when 7PLUS is invoked by another program (shell). -P Encoded files, error repors and correction files will be written with CR as line separator. This is necessary with those terminal programs, where 7PLUS-files must be sent using the unprotocolled binary transfer option, because it can't handle 8 bit characters in textmode. This is even more important on the AMIGA or under UNIX , because the standard line separator there is LF! Uploading a file in binary mode, that has LF as a line separator, would create a useless file on the other side, because LFs are alway stripped. The standard line separator in Packet Radio is CR! The result would be a file that contains one extremely long line! -T This option can be used to append a file termination string to encoded files, error reports and correction files. Example: 7plus test.txt -sp 2 -p -t /ex -Y Assume YES on all queries. This option is useful, when 7PLUS is invoked from another program (shell, server). -# This option is intended for use of 7PLUS in a server. When encoding, 7PLUS will create '7plus.fls', wich will contain the Page - 10 number of files that were created and the MS_DOS-compatible name, e.g.: 10 TEST A summary of command line options can be obtained by invoking 7PLUS without parameters. Tips: ===== Make sure your tnc/terminal combination is capable of handling the 7PLUS character set described above. It is important to set all parameters relating to 8 bit characters correctly (8BITCONV ON, AWLEN 8....). The streamswitch character should be set to $01 (CTRL-A), because 7PLUS uses all printable characters. Otherwise, unwanted streamchanges could occur, when uploading 7PLUS-files. Also, ensure that the RS232 link ist set to 8 data bits. If you still are not able to handle 7PLUS-files, get in touch with the author of your terminalprogram. since it is in his best interest that his program is able to cope with all eventuallities, i am convinced he will make sure it is updated. YAPP: ----- Some old YAPP-version will strip bit 8.... Get a newer one. Non-W0RLI(MSYS)/AA4RE/F6FBB/TheBox-BBSs: ---------------------------------- Some of these BBS-clones strip bit 8! This was reported to me from Hungaria. The Author of that BBS is working on it already. In the UK, there are BBSs around, that will strip all charachters in the range E0h to FFh. The Problem is already being investigated. Also, there are BBSs that strip the FEh charachter. This is no problem, because 7PLUS does not use this charachter. Warning: TheFirmware/WA8DED firmware ------------------------------------ Can't handle 7PLUS-files in terminalmode, unless patched or if TheFirmware v2.2a DAMA (NORD><LINK, Germany) or newer is used. Since this firmware is primarilly devised for host mode operation and should be used that way, this does not pose a problem. 7PLUS on AMIGA: --------------- Most terminal program for the AMIGA computer can't handle 7PLUS- files in textmode. This is because, most of them interpret the AMIGA control characters in the range 80h-9Fh. It would be wiser to Page - 11 display them in an appropriate way by replacing them by unproble- matic characters and not interpret them as control characters. Because of this fact, 7PLUS-files should be downloaded in binary mode. To be able to decode them, it is necessary to use the '-X'- option of 7PLUS to extract them from the logfile. Editing the files is not required then. When uploading 7PLUS-files in binary mode, the AMIGA line seperator LF must be replaced by the seperator required by packet: CR. This can be done using the '-p'-option of 7PLUS. How to download 7PLUS-files from a BBS: ======================================= Before downloading huge files from the BBS, you must make sure your terminalprogram and tnc are correctly configured. For this purpose you find ASC.TXT in the distribution archive of 7PLUS. Upload it to the BBS addressed to yourself and download it again. If it doesn't look like the original any more, check your configuration and read the chapter 'Tips:'. If the test went ok, then you're set to download 7PLUS-files. The simplest way to download 7PLUS-files is to open a capturefile and continuously write the received files into it. Note that you should not call the capturefile the same as the 7PLUS-files you download! Use a name like 'savefile' for instance. When you have finished downloading the files (did you think of looking for correction files that may already be in the BBS?), you must extract them from the capturefile. For this, an editor can be used or - much more convenient - 7PLUS itself! 7PLUS will not only find and extract the files from the capturefile, but also create the correct filenames for them. Invoking 7PLUS like this: '7plus savefile -x' will do the job of extracting the files from the capturefile. Now, the files are in the format that 7PLUS needs and you can begin decoding. Let's see, what we have on disk now: (This is only an example) TERM952.P01 TERM952.P02 TERM952.P03 TERM952.P04 TERM952.COR Ah, fine... You thought of downloading the correction file that was already in the BBS. Page - 12 There are two ways of decoding these files now: '7plus term952.cor' <-- Do the correction. '7plus term952.p01' <-- Decode. or '7plus term952' <-- 7PLUS does all by itself. Be careful about the second way, if a file called 'term952' already exists in the current directory, 7PLUS would then not try to decode 'term952.p01', but encode 'term952'! This is why i pointed out, that you should use a name for the capturefile, that has no relation to the name of the 7PLUS-files. If decoding went ok, 7PLUS will tell you: 'Decoding succesful! 'term952.exe' 43210 Bytes.' TERM952.EXE is the decoded file. What you have to do from now, depends entirely on the hints from it's originator. As the author of 7PLUS, i don't have anything to do with it anymore from this point on. Let's suppose decoding did not go allright. 7PLUS says: 'Decoding not successful. Send error report 'term952.err' to originator of 'term952.exe' for correction.' This just means, that you should take 'term952.err' and shove it...errr... i mean send it to the originator of 'term952.exe' and wait for the correction file to return from him. In the past it happened that OMs sent me error reports of programs I have never uploaded and requested the according correction files from me! Let me say this again in full clarity: The one who uploaded the files that were corrupted in store&forward is the one you have to send the error reports to!!! I, as the author of the encoder used, have nothing to do with it. Comparison: A forwarding agency delivers goods that were damaged on the transport. No-one in his right mind would have the idea to hold the builder of the truck used for the transport (Me) liable, when the truck (7PLUS) is ok, but the roads (the network) are not.... How to upload 7PLUS-files to a BBS: =================================== As in the previous chapter, make sure your terminal and tnc are configured correctly to handle 7PLUS-files. If you don't take all Page - 13 precautions, the files may not be decodable. So, avoid frustrating the other users an think before acting. You should write a small info in which you explain, what program is contained in the files you upload. State what system the program is for, what hardware is needed, and what compressor (LHARC, LHA, PKZIP) was used, and what files are in the archive and last but not least what encoder was used for the upload (7PLUS in this case). This way one can see right away, if the program is any use for him. The title of this info should be brief but yet state all needed info, e.g.: 'TERM952.EXE Info' When uploading to a general purpose board (ALL, SOFTWA..) you should also state the system the program runs on: 'TERM952.EXE Info (IBM)' OK, now you can do the encoding. Bear in mind that some BBSs restrict the size of bulletins. Ask your SYSOP about this. Take this into regard when splitting the file up ('-s' or '-sp' option). Let's say, TERM952.EXE has a size of 43210 bytes. When encoded, the size will increase by about 11% to 48k. Let's also say, the maximum size of the bulletins should be 4k. Then you should split into 12 parts using the '-sp' option or into blocks of about 4k using the '-s 60' option. '7plus term952.exe -sp 12' or '7plus term952.exe -s 60' OK, encoding is done. You should have the files TERM952.P01 through TERM952.P0C (note the hexadecimal numbering!!!!) on disk. The files can now be uploaded seperately. Here also, you should choose an informative title: 'TERM952.P01 / 0C (7PLUS) Means: TERM952 is encoded with 7PLUS and it's the first file of 12 (0C hex) parts. Even better, but somewhat cryptic to newcomers: 'TERM952.EXE.P01/0C.7PL' The best way to read this, is from the rear: it is a 7PLUS-file, the first of a 12 part upload, and the resulting program is called TERM952.EXE. After uploading all the files, you should check them by downloading them right away. They could have become corrupted on the way from your terminal to the BBS! In case they really were corrupted, generate a correction file and upload it. It is not really necessary to upload the entire file that was corrupted. Page - 14 And now: wait for error reports...... 7PLUS can be compiled on IBM-PC, ATARI-ST, AMIGA and UNIX-systems. See notes in the source code. 7PLUS is written for ham use and therefore costs nothing. It may be distributed and copied freely, but only in its entirety. That is including source code and DOCs. Use of 7PLUS in commercial surroun- dings is not allowed and no fee may be charged for 7PLUS. 73s, Axel in Bremerhaven, DG1BBQ @DB0CL.DEU.EU Page - 15